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ABSTRACT 



In 1986 the Kentucky General Assembly passed legislation 
directing the Legislative Research Commission to appoint a special 
subcommittee, the Special Subcommittee on Essential Skills Testing, to study 
the Kentucky Essential Skills Testing Program (KEST) and to make 
recommendations concerning the program. This report presents the findings and 
recommendations of this subcommittee. Kentucky began its mandated statewide 
testing program in 1978. The KEST provides information about each student's 
mastery of essential skills, or criterion-referenced data, and an estimate of 
how well Kentucky's students do in comparison with a national sample. When 
the KEST is considered in terms of the statutory requirements for testing, it 
seems to fulfill these purposes. However, it only provides an estimate of 
normative information, and is not as reliable at the individual student level 
as a longer norm- referenced test would be. The Subcommittee met six times 
during the year and heard the testimony of various experts and reviewed 
documents pertaining to the KEST and other state and nationally normed tests. 
As a result it made two recommendations: (1) the Kentucky Department of 

Education should administer a criterion-referenced test based on the 
essential skills in grades K-12, and also administer a nationally normed 
achievement test in grades 3,' 5, 7, and 10; and (2) Kentucky should 

participate in the 1988 Southern Regional Education Board/National Assessment 
of Educational Progress Testing Program. (SLD) 
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REPORT OF THE SPECIAL SUBCOMMITTEE 

ON 

ESSENTIAL SKILLS TESTING 

r> 

INTRODUCTION 

In 1986, the General Assembly passed HCR 132, directing the Legislative 
Research Commission to appoint a special subcommittee of the Interim Joint Committee 
on Education to study the Kentucky Essential Skills Testing (KEST) Program and to make 
recommendations concerning the program. In addition, the 1986 General Assembly passed 
a sunset provision for KEST effective at the end of the 1987-88 testing cycle. The 
Legislative Research Commission established the Special Subcommittee on Essential Skills 
Testing and appointed seven members. 

The purpose of this report is to outline goals and objectives of state testing pro- 
grams; review Kentucky’s testing program, its history and legislated requirements, ievicw 
large scale testing programs in other states; review national testing initiatives; summarize 
the subcommittee’s activities; and present the subcommittee’s recommendations. 

GOALS AND OBJECTIVES OF STATEWIDE TESTING PROGRAMS 

If the goal of a testing program is to compare an individual test score 
with the performance of others, a norm-referenced test is probably need- 
ed. If, on the other hand, the goal of the testing program is to compare a 
test score with a certain standard or level of mastery, a criterion- 
referenced test is needed . . . .Can one type test do it “all”? Probably 
not. But, it is important to recognize that the norm-referenced tests and 
criterion-referenced tests are designed for different purposes and must 
be interpreted differently. 1 

Dawson and Dawson summarized three general goals or objectives for testing 
Programs: 

1. Accountability. To evaluate school performance and to hold schools accoun- 
table for assuring basic student performance. 

2. Student Diagnosis. To diagnose individual student weaknesses and problem 
areas and direct them to specific remediation. 

I 

3. Curriculum evaluation. To review and evaluate the curriculum to see il it 
meets specified objectives. 2 

For the most part,*an individual state testing program cannot be labeled right or 
wrong. Basically, the testing program selected is a reflection of state policy. As long as the 
test is standardized, valid, reliable, and objective, and the results are interpreted within the 
constraints of the type test used, the program is defensible. 

O 

ERLC 
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KENTUCKY’S TESTING PROGRAM 

Kentucky began its mandated statewide testing programs with the passage of the 
Educational Improvement Act in 1978. The original wording of the intent section of the 
bill, codified as KRS 158.660, gave the following reasons for adopting a statewide testing 
program: 

It is the intention of the general assembly in enacting this legislation to 
assure the right of each student in the public schools of this state to ac- 
quire the basic knowledge and learning skills essential for completing 
high school, pursuing a course of study in postsecondary education, or 
entering the work force in our society. It is further the intention of the 
general assembly to assure each student in the public schools access to 
those programs and services appropriate to his educational needs in the 
areas of basic academic and learning skills development (HB 579 Section 
3). 

The bill required the Department of Education to purchase, or cause to be 
developed, tests for annually measuring student progress and achievement in the basic skills 
in grades 3, 5, 7, and 10 beginning in 1978-79. The Department chose to administer the 
Comprehensive Test of Basic Skills (CTBS) published by CTB/McGraw-Hill. 

KRS 158.660 was not changed substantially until passage of SB 202 in 1984, when 
the section was amended to read as follows: 

It is the intention of the general assembly in enacting KRS 158.650 to 
158.740 and this Act: 

(1) To assure the right of each student in the public schools of this state to 
acquire the competencies in the essential skills necessary for completing 
high school, pursuing a course of study in post-secondary education, or 
entering the work force in our society; 

(2) To assure each student in the public schools access to those programs 
and services appropriate to his educational needs in the areas of com- 
petencies in the essential skills; 

(3) To assure that each school district will be held accountable for students 
in meeting standards of attainment of competencies in the essential skills 
and for meeting standards in providing programs and services for 
students; 

(4) To require the state board of education to establish standards by which 
acceptable levels of attainment shall be determined; and 

(5) To require the state board of education to identify school districts which 
are educationally deficient and to provide such districts with program, 
financial, and other consultation and assistance as necessary to develop 
and implement strategies and programs to eliminate deficiencies. 
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The 1984 Legislature also began the Kentucky Essential Skills Testing Program, in 
order to “assure that each pupil in the public schools of the Commonwealth is taught and is 
mastering the essential skills necessary to function in each basic skills area ...” (KRS 
158.750(1)). The legislation called for testing in all grades for mastery of skills in 
mathematics and reading in 1984-85 and added testing of spelling, writing and library 
research reference skills in 1985-86. The Department contracted with CTB/McGraw-Hill 
for $1 .96 million to develop the customized tests. 

In the construction of the tests, CTB/McGraw-Hill has included additional items 
that, in combination with certain criterion-referenced items, yield estimated normative in- 
formation. Because these items form a shortened version of the norm-referenced test, 
CTBS-U, there is a larger degree of error in the estimated normative scores at the individual 
student level. But, as the number of students in the group increases, such as at the school 
level, the district level, or the state level, the estimated averages are very close to what the 
state would have had if every student had taken the CTBS-U. 

Thus, the KEST provides information about each student’s, or group of 
students', mastery of essential skills or criterion-referenced data and an estimate of how 
well Kentucky’s students do in comparison to a national sample or an estimate of norm- 
referenced data. 

The district test scores on the KEST are reviewed at the state level to determine 
whether they meet standards set by the State Board of Education. Districts not meeting the 
standards must develop and implement an educational improvement plan. Failure to make 
progress in correcting deficiencies can result in the district’s being declared educationally 
deficient. 

Another way the KEST scores are used at the state level is to allocate remediation 
funds to local school districts. The money is to provide special classrooms or teacher’s aides 
for those students who, at the end of kindergarten, first and second grades, fail to master 
the essential skills as measured by the KEST. 

When we consider Kentucky’s testing program in terms of the stated statutory 
purposes for testing, we might conclude that the KEST program fulfills those purposes to 
whatever extent the essential skills adequately represent the competencies necessary for 
completing high school, for pursuing a course of study in postsecondary education or for 
entering the work force. Student scores on the KEST are useful in meeting the requirements 
to the degree that the test measures mastery of the prescribed skills. 

In terms of Dawson and Dawson's classification presented earlier, the KEST pro- 
gram is used as an accountability measure, may be used for student diagnosis, and may 
serve as a tool in curriculum evaluation, to the extent that the test data may indicate cur- 
riculum strengths and weaknesses for skills. 

As previously stated, the KEST may be used as a tool for diagnosing individual 
strengths and weaknesses. The timing of testing may not facilitate this use, however. For 
example, a student in the third grade takes the test at the end of the school year. The scores 




6 



Research Memorandum No. 435 

Page Four 

are sent back to the district in late summer. The individual scores may be given to the third 
grade teacher, but she no longer has the student. The scores may be given to the fourth 
grade teacher, but she is responsible for teaching a different set of skills, though she may 
teach those third grade skills which are necessary for learning the fourth grade skills. 
Neither way of reporting scores maximally facilitates student learning at the individual 
level. If the district’s students have weaknesses which manifest themselves at the district 
level, then they are addressed through the educational improvement plan process. 

The KEST generally meets the statutory requirements in terms of school district 
accountability, but the statute speaks to each student’s having the skills to pursue 
postsecondary opportunities. Kentucky students will frequently find themselves competing 
with people from other states in higher education, advanced training, and in the job 
market. Therefore, an argument can be made for a norm-referenced test. The KEST pro- 
vides estimated normative information, but is not as reliable at the individual student level 
as a longer norm-referenced test would be. 

TESTING PROGRAMS IN OTHER STATES 

Roeber has described the large-scale testing programs in forty-four states, two 
Canadian provinces, and Guam. 3 In reviewing these programs, it is apparent that the states 
have chosen many different approaches to testing programs. Some states have criterion- 
referenced testing only, others have norm-referenced testing only, some administer a 
customized achievement test, and still others administer several types of tests. 

Most states test selected grades only. Ohio permits the local district to determine 
which grades will be tested within certain parameters. Most states test only one high school 
grade. Those testing more than one year at the high school level generally have an exit ex- 
amination requirement and offer retesting opportunities at other grades beyond the first 
time the test is given. Kentucky is the only state which requires criterion-referenced testing 
at all grades, while Arizona is the only state other than Kentucky which requires a norm- 
referenced test in grades 1-12. Virginia has criterion-referenced assessment resource 
material available for grades K-12 in nine areas, but the Board of Education requires that 
the school divisions use the resource materials or approved alternatives for language arts 
and mathematics in grades 1 -6 only. Use at all other grade levels and other subject areas are 
local options. 

Another difference among state testing programs is whether all students in a par- 
ticular grade are tested or merely a sample of students in certain grades is tested. Most 
states test all students in the designated grades. 

Shelf tests, customized state tests, and a combination of both are found among 
the testing programs. Customized tests may be developed with the assistance of a test 
publishing company, teachers, university faculty, or advisory panels. 

Subject areas most commonly tested are mathematics, reading, and language arts. 
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Several states require a writing sample for certain grades. Other subjects less frequently 
tested are science, social studies, and citizenship. Kentucky is the only state to designate 
spelling and library research reference skills specifically, although in other states these areas 
are probably covered to some extent on the language arts tests. 

NATIONAL TESTING INITIATIVES 

The Southern Regional Education Board (SREB) joined forces with the federally- 
funded National Assessment of Education Progress (NAEP) testing program in an attempt 
to measure student progress in response to education reforms in the South. The program 
provides its participants current national and state data as each state tests a representative 
sample of its students at each testing. Tests in one or two subject areas are administered to a 
scientifically derived sample of eleventh grade students in each state. The first year reading 
was tested; the second year writing was added. This year mathematics and U.S. History will 
be assessed. 

All states participate in NAEP, but this particular program gives state data and 
permits regional comparisons. Eight SREB states elected to participate, at a cost of 
$40,000. The tests are not tied to a curriculum or a textbook series, but are NAEP items 
agreed to by the participating states. 

NAEP tests samples of students selected as representative of the nation as a whole 
at three age levels— 9, 13, and 17. Results are reported for such subgroups as sex, race, and 
region of the country. The U.S. Department of Education and the National Science Foun- 
dation have recently funded a grant to the Council of Chiel State School Officers to form a 
consortium of educators to develop recommendations for a state-by-state assessment ol 
student achievement, with the goal being some state-level NAEP testing in 1990. 

SUBCOMMITTEE ACTIVITIES 

The subcommittee met six times during the interim. At its first meeting, the 
Associate Superintendent of the Office of Research and Planning presented the 1985-86 
KEST results. That school year nearly 88% of Kentucky’s students mastered the essential 
skills. Sixty-three percent of the students did as well as or better than the average student in 
the national norm group. In almost every category, the 1985-86 students performed better 
than those' of the previous year. Ms. Brown recommended expansion of the remediation 
program. The test results also suggested that elementary students were performing better 
than high school students. 

The subcommittee heard testimony from representatives of several educational 
organizations at its second meeting. The president of the Kentucky Education Association 
emphasized that achievement tests are not an end, but a means to an end. He suggested that 
tests are an education tool, which, in combination with professional judgment, adequate 
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and up-to-date materials, reasonable class sizes, and student and community commitment, 
can encourage a quality education for all students. 

The Legislative Chairman of the Kentucky Congress of Parents and Teachers sug- 
gested that teachers were not adequately prepared to interpret test results and were 
therefore unable to explain them adequately to parents. She also questioned the reliability 
and validity of the KEST. In addition to reliability and validity studies, she recommended a 
cultural bias review. 

The Executive Director of the Kentucky Association of School Administrators 
reported that many superintendents were concerned about the validity of the KEST, the 
emphasis being placed on district rankings and the late date lor test administration. He 
recommended further reliability and validity studies on the KEST. 

A professor in the Department of Education Psychology and Counseling at the 
University of Louisville discussed his concerns with the testing program at the subcom- 
mittee’s third meeting. In his opinion, the decision to develop a Kentucky test for essential 
skills was probably a mistake, since the underlying assumption is that the skills students 
need to know in Kentucky are different from those required for students in other states. He 
was critical of the Department of Education’s decision to combine a shortened version of 
the CTBS with the KEST, in that there are not enough items to do all the things the test is 
intended to do. He was also critical of the Boston College study, because it failed to com- 
pare the KEST to any absolute standard, but instead compared it to standards taking into 
consideration the difficulty of the task. He suggested that the real question is whether the 
KEST should have replaced the CTBS. 

At its fourth meeting the subcommittee heard a report on the Southern Regional 
Education Board/National Assessment of Education Progress Testing Project 
(SREB/NAEP), presented by an SREB staff member. He noted that SREB’s interest is 
primarily in how test scores and test information can be used by educators and government 
officials to help shape and change educational policies. 

In an effort to evaluate educational reform, SREB began looking at testing and 
test results four years ago. They found that the national tests were only used in a few states 
and that, of the fifteen SREB states, only three were using a national test. They also found 
that the national averages or norms used were seven or eight years old, and that there was 
no way for one state to compare its performance with that of another state. The 
SREB/NAEP program began in an attempt to provide current information, national com- 
parisons and state-to-state comparisons. 

The states participating in the SREB/NAEP project administer tests in one or two 
subject areas to a scientific sample of students. Only state data are generated. The tests are 
not tied to a particular curriculum or textbook series and include a wider range of questions 
than is found on most tests. The tests are not meant to replace criterion-referenced tests. 

Kentucky does not presently participate in the project, but the Deputy Associate 
Superintendent of the Office of Research and Planning indicated that the testing stall was 
recommending participation in the 1988 testing cycle. 
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At the fourth meeting the Superintendent of the Calloway County School District 
and an Associate Professor of Counseling at Murray State University discussed their fin- 
dings from an empirical study involving administration of both the KEST and CTBS. They 
found that the estimated norm scores of the KEST were not accurate predictors of CTBS 
scores and that the KEST was easier than the CTBS. Thus, a key problem from an ad- 
ministrator’s viewpoint is lack of confidence in the results. The superintendent recommend- 
ed making the KEST a criterion-referenced test only and administering it every two or three 
years, or giving a recognized norm-referenced test every two or three years. In the event it is 
decided to administer one test to all Kentucky students, he recommended a recognized, na- 
tionally normed achievement test. 

A school psychometrist for Calloway County discussed her concerns about the 
confusing individual test records, the fact that the test does not yield a vocabulary score, 
the turnaround time for scoring and the inability to hand score the KEST hindering work 
with new students. 

The superintendent and a psychologist/psychometrist from Owensboro Indepen- 
dent School District criticized the test for artificially inflating test scores. The former sug- 
gested a norm-referenced test, using as current norms as possible and administering dif- 
ferent forms of a test from year to year or changing tests every two or three years. 

The latter questioned whether the norm-referenced portion of the KEST and the 
CTBS measure the same kinds of student achievement. He also questioned how well the 
KEST measures students’ performance on the prescribed essential skills. 

At its fifth meeting the subcommittee heard testimony from a representative of 
the Boston College Center for the Study of Testing, Evaluation and Education Policy, 
outlining the Center’s findings and recommendations from its study of the mathematics 
and reading portions of the KEST. Generally, the study found that CTB/McGraw-Hill did 
a thorough and professional job in producing the twenty-six tests and that with only two ex- 
ceptions the comparability of the KEST norm-referenced score estimates to those that 
would have to be obtained with the CTBS is adequate for the intended use of the results. 

In response to a question concerning the best way to find out if Kentucky s 
students are learning the essential skills and to compare Kentucky’s students to students in 
other states, he recommended two separate tests. First, test selected grades using a stan- 
dardized norm-referenced, commercially available test. Second, develop a criterion- 
referenced test specifically measuring the essential skills, ideally using existing items that 
have been screened for measuring the skill. He also recommended that the test be limited to 
a multiple choice format and that more items be devoted to each skills. Finally, he caution- 
ed against placing too much emphasis on testing. 

At the subcommittee’s sixth and final meeting, a representative of the Department 
of Education presented the 1986-87 KEST results. The percent of students mastering the 
essential skills was 88.8 in 1987, as compared to 87.7 in 1986. There was a higher percentage 
of students mastering reading in each grade, kindergarten through eight; however, there 
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was a slight decline in the percent of students mastering the reading essential skills in grades 
nine through twelve. This is the second year in a row for such a decline in these grades. 

The national comparison data show a slight improvement in three out ol lour 
grades, with the tenth grade showing a slight decline from 1986. 
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RECOMMENDATIONS 

The subcommittee recommends and submits the following to the Legislative 

Research Commission and the Interim Joint Committee on Education: 

1) That the Kentucky Department of Education administer a criterion-referenced test 
based on the essential skills in grades K-12 and also administer a nationally normed 
achievement test in grades 3, 5, 7 and 10; and 

2) That Kentucky participate in the 1988 Southern Regional Education Board/Na- 
tional Assessment of Educational Progress Testing Program. 
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FOOTNOTES 



I. Margaret Jorgensen, Basic Differences Between Norm-Referenced and Criterion- 
Referenced Tests (Atlanta: Southern Regional Education Board, 1986), pp. 3-4. 

2. Karen S. Dawson and Richard E. Dawson, Minimum Competency Testing and Local 

Schools(St. Louis: Washington University, 1985), pp. 14-15. 

3. Edward Roeber, Large Scale Assessment Programs: Program Descriptions (Lansing: 

Michigan Department of Education), pp. 1-15 and attachments. 
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